-
Notifications
You must be signed in to change notification settings - Fork 603
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
refactor(sql): add a base SQLGlot
backend for DuckDB
and ClickHouse
#7796
Conversation
d174f3e
to
f3ee1a6
Compare
ACTION NEEDED Ibis follows the Conventional Commits specification for release automation. The PR title and description are used as the merge commit message. Please update your PR title and description to match the specification. |
8a5c21f
to
76f516a
Compare
df63f39
to
593ec6a
Compare
c00c30b
to
03e86bd
Compare
@@ -30,7 +30,7 @@ def test_tpc_h01(lineitem): | |||
avg_qty=t.l_quantity.mean(), | |||
avg_price=t.l_extendedprice.mean(), | |||
avg_disc=t.l_discount.mean(), | |||
count_order=t.count(), | |||
count_order=lambda t: t.count(), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@kszucs Still not 100% sure whether this breakage is something we want.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this still failing? We can certainly fix this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yep, if I revert this change then the test fails with an integrity error.
fa3f290
to
b733e08
Compare
7f0c102
to
f30e7ad
Compare
ibis/backends/clickhouse/tests/snapshots/test_select/test_array_join_in_subquery/out.sql
Outdated
Show resolved
Hide resolved
ibis/backends/clickhouse/tests/snapshots/test_select/test_join_self_reference/out.sql
Show resolved
Hide resolved
ibis/backends/clickhouse/tests/snapshots/test_select/test_named_from_filter_groupby/out1.sql
Show resolved
Hide resolved
it's alive! tests run (and fail) chore(duckdb): naive port of clickhouse compiler fix(duckdb): hacky fix for output shape feat(duckdb): bitwise ops (most of them) feat(duckdb): handle pandas dtype mapping in execute feat(duckdb): handle decimal types feat(duckdb): add euler's number test(duckdb): remove duckdb from alchemycon feat(duckdb): get _most_ of string ops working still some failures in re_exract feat(duckdb): add hash feat(duckdb): add CAST feat(duckdb): add cot and strright chore(duckdb): mark all the targets that still need attention (at least) feat(duckdb): combine binary bitwise ops chore(datestuff): some datetime ops feat(duckdb): add levenshtein, use op.dtype instead of output_dtype feat(duckdb): add blank list_schemas, use old current_database for now feat(duckdb): basic interval ops feat(duckdb): timestamp and temporal ops feat(duckdb): use pyarrow for fetching execute results feat(duckdb): handle interval casts, broken for columns feat(duckdb): shove literal handling up top feat(duckdb): more timestamp ops feat(duckdb): back to pandas output in execute feat(duckdb): timezone handling in cast feat(duckdb): ms and us epoch timestamp support chore(duckdb): misc cleanup feat(duckdb): initial create table feat(duckdb): add _from_url feat(duckdb): add read_parquet feat(duckdb): add persistent cache fix(duckdb): actually insert data if present in create_table feat(duckdb): use duckdb API read_parquet feat(duckdb): add read_csv This, frustratingly, cannot use the Python API for `read_csv` since that does not support list of files, for some reason. fix(duckdb): dont fully qualify the table names chore(duckdb): cleanup chore(duckdb): mark broken test broken fix(duckdb): fix read_parquet so it works feat(duckdb): add to_pyarrow, to_pyarrow_batches, sql() feat(duckdb): null checking feat(duckdb): translate uints fix(duckdb): fix file outputs and torch output fix(duckdb): add rest of integer types fix(duckdb): ops.InValues feat(duckdb): use sqlglot expressions (maybe a big mistake) fix(duckdb): don't stringify strings feat(duckdb): use sqlglot expr instead of strings for count fix(duckdb): fix isin fix(duckdb): fix some agg variance functions fix(duckdb): for logical equals, use sqlglot not operator fix(duckdb): struct not tuple for struct type
62eff1d
to
be26c52
Compare
SQLGlot
backend based on the new relational operations and implementations for DuckDB
and ClickHouse
SQLGlot
backend for DuckDB
and ClickHouse
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1
Great refactor!
Rewrite the DuckDB and ClickHouse backends using a new base
SQLGlotBackend
and associated compiler.